
THE ASPEN INSTITUTE 



Assessment 2.0: “Next-Generatien” 
Cemprehensive Assessment Systems 



An Analysis of Proposals by the 

Partnership for the Assessment of 
Readiness for College and Careers 

and 

SMARTER Balanced Assessment Consortium 



By Joaquin R. Tamayo, Jr. 

December 2010 



THE ASPEN INSTITUTE 




© The Aspen Institute Education & Society Program 2010 



Introduction 



On September 2, 2010, the U.S. Department of Education announced the winners of the $350 
million Race to the Top Comprehensive Assessment Systems Competition: the Partnership for 
the Assessment of Readiness for College and Careers (PARCC) and the SMARTER Balanced As- 
sessment Consortium (SMARTER). In his announcement, Secretary of Education Arne Dun- 
can asserted that these “next-generation” assessments are “an absolute game-changer in public 
education.” Designed to replace individual state tests in English language arts and mathematics 
currently mandated by the No Child Left Behind Act (NCLB), the PARCC and SMARTER assess- 
ment systems will usher in a new and different approach to assessment system design to comple- 
ment the adoption and implementation of the Common Core State Standards (CCSS). These new 

standards and assessments have significant implications for how 
states and districts organize and support the challenging work of 
principals and teachers to improve student outcomes. 

By the 2014-2015 school year, for the first time in the history of 
American education, the daily work of teachers in PARCC and 
SMARTER consortia states will be driven by common standards 
and assessments designed to prepare all students for success in 
college and the workplace. States and districts need to build a 
“next-generation” policy infrastructure to ensure that school- 
based professionals have the resources they need to take full 
advantage of the PARCC and SMARTER assessment systems. 

This side-by-side comparative brief describes the system proposed by each consortium in order to 
assist state and district-level leaders in planning for implementation. 

As per the requirements articulated by the U.S. Department of Education, both the PARCC and 
SMARTER approaches to comprehensive assessment system design are grounded in four basic 
principles: 

1. Assessments are common across states and aligned to the CCSS. Students in grades 3-11 
in PARCC states and students in grades 3-8 and 11 in SMARTER states will take common 
assessments aligned to the CCSS in English Language Arts (ELA) and mathematics, including 
English learners and students with special needs. Eurther, common achievement standards 
and cut scores will allow for cross-state comparability of student performance between con- 
sortium states. 

2. Students take “performance-based” assessments for accountability. Both the PARCC and 
SMARTER systems will require students to demonstrate their skills in reading, writing, and 
mathematical reasoning on higher-order tasks, including research and essay-writing, in order 
to measure students’ readiness for college and careers. 



States and districts need to 
build a “next-generation” 
policy infrastructure to 
ensure that school-based 
professionals have the 
resources they need to take 
full advantage of the 
assessment systems. 
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3. The assessment systems are “computer-based” for more sophisticated design and quick, 
reliable scoring. With a few exceptions, the assessments, including many of the performance 
tasks, will be delivered and scored using computer and online resources, which will increase 
overall system functionality and dramatically reduce the amount of time that schools and 
teachers must wait for student achievement and growth data. 

4. Transparent reporting systems drive effective decision-making. The PARCC and SMART- 
ER systems will provide clear and comprehensible student performance data to all stakehold- 
ers, from state and district leaders to teachers and parents. Student achievement and growth 
reports will also indicate whether students are “on track” to meet college and career-readiness 
expectations, thus facilitating responsive intervention where necessary. 

By grounding their system designs in these four principles, aggregating the design and field-test- 
ing to a large scale, and taking advantage of federal resources, PARCC and SMARTER promise 
to reverse the widespread decline in test quality and rigor that has been documented over the last 
several years.^ These new assessment systems will replace the NCLB-mandated assessments cur- 
rently used in participating states. 

Though similar in many respects, there are at least four important differences between the 
PARCC and SMARTER approaches to system design that warrant close inspection: 

1. Summative Assessments: PARCC envisions a “though-course” model of summative assess- 
ment for accountability, which will require participating states to administer quarterly assess- 
ments in both ELA and mathematics. Each of these quarterly assessments will be included in 
summative scores for accountability determinations. SMARTER sticks to a more traditional 
“end-of-year” approach, with summative assessments administered during the last 12 weeks 
of the school year. 

2. High School Assessments: PARCC requires tests in grades 9-11, while SMARTER requires 
testing only once in high school (in the 11th grade). The SMARTER consortium will develop 
assessments for grades 9 and 10, but these assessments are optional for states. 

3. Computer-adaptive versus Computer-based: SMARTER assessments will be computer- 
adaptive — a method of test administration that adjusts in real time an assessments level of dif- 
ficulty based on individual students’ responses. PARCC assessments will be computer-based 
but adhere to a single form for all students. 

4. Setting Cut Scores: SMARTER will set initial cut scores in the summer of 2014, following the 
field testing of its assessments but before the first full-scale administration of the assessment 
system in all consortium states. Alternatively, PARCC will finalize assessment standards and 
set cut scores only after full-scale administration in 2014-2015. 



1 “No Child Left Behind Act: Enhancements in the Department of Educations Review Process Could Improve State Academic Assessments,” U.S. 
Government Accountability Office, GAO-09-911, September 24, 2009. 
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This comparative brief describes the major elements of each assessment system and is designed 
to assist system leaders in gearing-up for implementation. This analysis highlights the following 
elements of the PARCC and SMARTER proposals: 

• Consortia Governance and Membership 

• Assessments for Accountability 

• Additional Assessments Not for Accountability 

• Assessment Descriptions 

• Technology and Capacity Requirements 

• Implementation Timelines 

Individual states and districts need to understand these system-level decisions as they begin the 
process of determining how their existing systems (e.g. curriculum and instruction, information 
technology, budgeting and procurement) will be aligned to support effective implementation. 
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Consortia Govornanco and Momborship 



The PARCC and SMARTER consortia are voluntary membership organizations composed of gov- 
erning and participating/advisory states (non-governing states are referred to as “participating” 
in PARCC and “advisory” in SMARTER). States are free to choose the consortium with which 
they are affiliated and whether they would like to act as governing or participating/advisory states. 
However, to remain or to become a member state of either consortium, a state must have adopted 
the CCSS by December 31, 2011, and each state must decide no later than the 2014-2015 school 
year which comprehensive assessment system it will implement, thus restricting states to mem- 
bership in only one consortium. 

Governing states are those which have fully committed to implementing their respective consor- 
tium’s assessment system by the 2014-2015 school year. Each governing state controls one vote on 
either the Governing Board (PARGG) or the Steering Gommittee (SMARTER), which are the main 
policy-making bodies for each consortium. Participating or advisory states, which at this point in 
time can belong to one or both of the consortia, will engage at some level in consortium activities 
but have not yet committed to implementing one particular assessment system. The table below 
identifies current consortia membership, with participating/ advisory members of both consortia 
marked in bold, as well as the lead project management organization for each consortium. 



Presently, 44 states and the District of Golumbia are affiliated with at least one consortium. 

The six states that are not members of either consortium are Alaska, Minnesota, Nebraska, Texas, 
and Virginia. 

Table 1: Consortia Membership [as of Dec. 2010) 



STATUS 


PARCC 


SMARTER 






Connecticut 






Hawaii 




Arizona 


Idaho 




Arkansas 


Kansas 




District of Coiumbia 


Maine 




Fiorida (fiscai agent) 


Michigan 




Georgia 


Missouri 




iiiinois 


Montana 


Governing States 


Indiana 


Nevada 




Louisiana 


New Mexico 




Maryiand 


North Carolina 




Massachusetts 


Oregon 




New York 


Utah 




Rhode Isiand 


Vermont 




Tennessee 


Washington (fiscal agent) 






West Virginia 






Wisconsin 






Alabama 




Alabama 


Colorado 




California 


Delaware 




Colorado 


Iowa 




Delaware 


Kentucky 




Kentucky 


New Hampshire 


Participating or 


Mississippi 


New Jersey 


Advisory States 


New Jersey 


North Dakota 




North Dakota 


Ohio 




Ohio 


Oklahoma 




Oklahoma 


Pennsylvania 




Pennsylvania 


South Carolina 




South Carolina 


South Dakota 






Wyoming 


Proiect Manaaement 


Achieve, Inc. 


WestEd 




Assessments for Accountability 



Perhaps the most significant difference between the PARCC and SMARTER proposals is their ap- 
proach to assessment for accountability. 

PARCC takes what it calls a “distributed” approach to assessment for accountability Assessments 
will be distributed throughout the school year and will together produce a summative score in 
EL A and mathematics for each student in grades 3-11. The individual weights of each assessment 
have not yet been determined. Students’ combined summative scores will indicate absolute levels 
of achievement against the CCSS at every grade level as well as progress (i.e. growth) toward col- 
lege and career readiness. 

SMARTER’s “balanced” approach to assessment combines end-of-year, computer-adaptive, sum- 
mative assessments for accountability with optional, computer- delivered interim/benchmark and 
formative assessments that will not be used for accountability. The end-of-year summative assess- 
ments also include standardized “performance events.” Unlike PARCC, which requires assess- 
ments in grades 3-11, SMARTER requires testing in grades 3-8 and only once in high school, at 
grade 11. SMARTER assessments for grades 9 and 10 that assess progress toward grade 11 college 
and career- readiness goals are available but optional for states. 



Table 2: Assessments fer Acceuntability 



ASSESSMENTS 

FOR PARCC’S “DISTRIBUTED” SYSTEM SMARTER’S “BALANCED” APPROACH 

ACCOUNTABILITY 



Two (2) assessments each year: 



Three (3) assessments each year: 



* End-of-year Literacy Assessment (ELA-4) 

• End-of-Year Mathematics Assessment (Math- 
4} 



• Summative Assessment in Reading 

• Summative Assessment in Writing, Listening and 
Speaking, and Language 

• Summative Assessment in Mathematics 



END-OF-YEAR 



Notable features include: 

o Computer-based assessments 

o Administered after roughly 90% of 
instruction 

o Includes “next-generation", computer- 
enhanced items 

o 1 00% computer-scored; one (1 ) week 
turnaround time for results 



Notable features include: 

o Computer-adaptive assessments 

o Two (2) ELA and two (2) math “performance 
events” in grades 3-8 

o Up to six (6) ELA and six (6) math performance 
events by grade 1 1 

o Summative assessments in grades 9-10 are 
optional for states 

o Administered within a 12-week window at the 
end of the school year 

o Combination of computer and human scoring; 
two (2) week turnaround time for results 



Six (6) assessments each year: 

• Focused Literacy Assessments — Writing From 
Sources (ELA-1 &2) 

• Extended Research/Writing Assessment 
(ELA-3) 



And 



THROUGH-COURSE 



Focused Assessments ofEssentiai Topics 
(Math-1 & 2) 

Extended Mathematics Assessment (Math-3) 



NOT APPLICABLE 



Notable features include: 

o Administered after roughly 25%, 50%, and 
75% of instruction, respectively 
o Includes a mix of constructed response 
and performance tasks 
o Combination of computer and human 
scoring; one (1 ) to two (2) week 
turnaround time for results 




Additional Assessments Not for Accountability 



The following table identifies the elements of the PARCC and SMARTER assessment systems that 
will not be used for accountability purposes. 

PARCC requires states to administer the Speaking and Listening Assessment (ELA-5) each year to 
all students in grades 3-11. However, a student’s score on this assessment will not be figured into 
his or her cumulative ELA score for the year. 

SMARTER plans to provide member states the option of accessing a full complement of Interim/ 
Benchmark Assessments in ELA and Math in grades 3-11. These interim/benchmark assessments 
are optional, will be administered as per state/district policy, and will not impact students’ yearly 
cumulative scores. 



Table 3: Additional Assessments Not for Accountability 



ADDITIONAL 

ASSESSMENTS 


PARCC’S “DISTRIBUTED” SYSTEM 


SMARTER’S “BALANCED” APPROACH 




One (1) assessment per year: 


Multiple assessments per year: 




• Speaking and Listening Assessment 
(ELA-5) 


• Interim/Benchmark Assessments in ELA 
and Math 


THROUGH- 

COURSE 


Notable features include: 

o Administered foilowing the ELA-3 
assessment, after approximateiy 75% 
of instruction 

o Teacher-scored using consortia- 
developed rubric; immediate 
turnaround time for results 


Notable features include: 

o Computer-adaptive assessments 

o To be implemented throughout the 

year as per state/district policy 
o Includes teacher-administered 
performance events 

o Combination of computer and human 
scoring; two (2) week turnaround 
time for results 

o Scoring will occur on the same scale 
as summative assessments for 
accountability 

o Items and tasks will be non-secure 
for teacher and principal analysis 


FORMATIVE 


Both PARCC and SMARTER will provide states the option of accessing a broad range of formative 
assessments and tools that teachers can use to diagnose student learning needs throughout the 
year. Many of these formative assessments will be computer-based and web-delivered and will 
also be useful in preparing students for interim and end-of-year assessments. 
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Assessment Descriptions 



The following are brief descriptions of the various assessments proposed by each consortium. 

PARCC 

English Language Arts: 

• End-of-year Literacy Assessment (ELA-4): This end-of-year computer-based assessment will 
include 40-65 “computer-enhanced” items to assess higher-order skills in reading, writing, 
and language/vocabulary This assessment is a single-session exam that will be administered 
after roughly 90% of instruction for the academic year. 

• Eocused Literacy Assessments — Writing Erom Sources (ELA-1 & 2): These “through-course” 
computer-based assessments will require students to complete constructed-response tasks 
through written analysis of selected texts. These are single-session exams that will be admin- 
istered after approximately 25% and 50% of instruction, respectively, for the year. 

• Extended Research/Writing Assessment (ELA-3): This “through-course” computer-based as- 
sessment requires students to read, evaluate, and select information from digital sources in 
order to compose an essay or research project. This multi-session exam will be administered 
after approximately 75% of instruction for the academic year. 

• Speaking and Listening Assessment (ELA-5): This “through-course” exam will require students 
to present their findings from their ELA-3 exam to classmates. Teachers will administer 
this exam and score students’ performance using a common rubric. This is a single-session 
exam, and while its administration by states is required, it will not be used for accountability. 

Mathematics: 

• End-of-Year Mathematics Assessment (Math-4): This end-of-year computer-based assessment 
will incorporate 40-65 “computer- enhanced” items to assess higher-order skills in math- 
ematical computation, reasoning, and modeling. This assessment is a single-session exam 
that will be administered after roughly 90% of instruction for the academic year. 

• Eocused Assessments of Essential Topics (Math-1 & 2): These “through-course” computer- 
based exams will test students’ understanding of specific CCSS math standards or clusters 
of standards that are the foundation for more advanced math education. These are single- 
session exams that will be administered after approximately 25% and 50% of instruction, 
respectively, for the academic year. 

• Extended Mathematics Assessment (Math-3): This “through-course” computer-based exam 
will require students to demonstrate conceptual understanding, procedural fluency, and 
reasoning ability through the application of mathematical tools using complex and novel 
computer-based simulations. This multi-session exam will be administered after approxi- 
mately 75% of instruction for the academic year. 

SMARTER 

• Summative Assessment in Reading 

• Summative Assessment in Writing, Listening and Speaking, and Language 

• Summative Assessment in Mathematics 
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• These end-of-year computer-adaptive tests will include writing prompts, constructed 
responses, technology-enhanced items, and standardized performance events to as- 
sess students across the whole range of the CCSS in ELA and math in grades 3-8 and 
11. These single-session, computer- adaptive exams, along with the performance events, 
must be administered within a 12-week window at the end of the instructional year. 



Technology and Capacity Reqnirements 



The PARCC and SMARTER consortia expect that most assessments and tools will be computer- 
based and Web -delivered, requiring large-scale planning and coordination within and across 
every level of the education system. System leaders need to consider what these changes mean for 
their states and districts well in advance of the 2014-2015 implementation deadline. 

The PARCC and SMARTER consortia plan to work with states to ensure that schools have the 
technology they will need to implement the assessment systems. This assistance will focus on 
systems needed to implement the assessments for accountability. Additional, optional elements 
not for accountability will be implemented only in states and districts with the resources to do so. 
Therefore, what “full implementation” actually looks like by 2014-2015 will depend on state and 
district budgets. The table below details technology and capacity demands in four areas: data man- 
agement; assessment administration; curriculum and instruction; and professional development. 

Table 4: Technology and Capacity Requirements 



SYSTEM 

ELEMENTS 


PARCC 

will require states/districts to: 


SMARTER 

will require states/districts to: 


Data 

Management 


• Align existing data management systems with 
the Interactive Data Tool, a cross-consortia 
"open-source” system that houses student 
performance-related data 

• Generate three (3) different periodic reports: 

o Periodic Feedback Reports 
o Annual Stakeholder Reports 
o Item Analysis Reports 

• Provide appropriate logon access to student 
performance data to school-based 
professionals, parents, and students 


• Align existing data management systems with 
the SBAC System Portal, a cross-consortium 
“open-source” system that houses student 
performance-related data 

• Generate two (2) different periodic reports: 

o Performance Summaries for individual 
students 

o Class, School, District, and State 
Summaries 

• Provide appropriate logon access to student 
performance data to school-based 
professionals, parents, and students 


Assessment 

Administration 


• Administer and score assessments for 
accountability using computer-based and 
online resources. (Note: Assessments for 
grades 3-5 will not be administered online.) 

• Administer assessment system training for 
school-based professionals and students 
using computer-based and online resources 


• Administer and score assessments for 
accountability using computer-adaptive and 
online resources 

• Administer assessment system training for 
school-based professionals using computer- 
based and online resources, especially in the 
area of administering and scoring performance 
events 


Curriculum and 
Instruction 


• Provide school-based professionals access to 
the Partnership Resource Center, an online 
portal that will house model curriculum 
frameworks, released assessment items and 
tasks, and formative tools 

• Provide school-based professionals access to 
the Test Complexity Diagnostic Tool that will 
assess students’ reading levels and suggest 
appropriately-leveled texts 


• Provide school-based professionals access to 

the SBAC Educator Portal, an online system 
that will house student performance data, 
model instructional units, instructional and 
intervention recommendations, scoring 
rubrics, and an online, cross-consortium 
network for teacher-to-teacher communication 
and sharing of resources 


Professional 

Development 


• Train school-based professionals in system 
functionality including assessment 
administration, scoring, use of data, and 
appropriate use of online tools and resources 


• Leverage existing professional development 

networks and programs to train school-based 
professionals in system functionality including 
assessment administration, scoring, and 
appropriate use of online tools and resources 




Implementation Timelines 



As per U.S. Department of Education requirements, PARCC and SMARTER consortia states 
must implement assessment systems no later than the 2014-2015 school year. The following table 
details assessment system implementation timelines and tasks, which will be managed at the con- 
sortium level by PARCC and SMARTER’s project management organizations. Achieve, Inc. and 
WestEd. PARCC and SMARTER envision very similar implementation processes; however, there 
are some important operational differences, especially around the issues of standards-setting and 
establishing cut scores for the first full operational administration in 2014-2015. 



Table 5: Implementation Timelines 



YEAR 


PARCC 


SMARTER 


2010-2011 


• CCSS content analysis 

• Assessment blueprint development 

• Assessment item development 

• Performance task development 

• Draft accommodations manual for 
English language learners and students 
with special needs 


• CCSS content analysis 

• Learning progression development 

• Assessment protocol and training 
materials development 

• Assessment item development 

• Performance event development 


2011-2012 


• Implementation framework 
development 

• Pilot testing of select components of the 
assessment system 

• Pilot testing to include English language 
learners and students with disabilities 


• Pilot testing of select components of 
the assessment system 

• Pilot testing to include English 
language learners and students with 
disabilities 


2012-2013 


• Field testing begins in each consortium 
state 

• Data and CCSS-alignment review of 
test items 


• Field testing begins in each 
consortium state 

• Data and CCSS-alignment analysis 
of test items 


2013-2014 


• Field testing continues 

• Finalize accommodations manual; each 
state must adopt by end of year 


• Field testing continues 

• Initial standards and cut scores 
determined in Summer 2014 
following field testing 


2014-2015 


• Implementation of PARCC system at 

scale in all consortium states 


• Implementation of SMARTER system 

at scale in all consortium states 


2015 


• Standard-setting following first full 
operational administration 

• Initial cut scores determined 


• Final standard-setting following first 
full operational administration 

• Cut scores validated using 2014- 
2015 data 
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Conclusion 



The adoption of the Common Core State Standards and the new comprehensive assessment 
systems create important responsibilities and opportunities for system leaders. In order to be 
prepared for the successful implementation of the comprehensive assessments by the 2014-2015 
school year, system leaders need to plan for transitions that involve many aspects of their organi- 
zations and new interactions across states and districts. Questions to be considered include: 

Technology 

What technology infrastructure is needed to administer the new computer-based assessment 
systems? Are computer-based assessments prevalent now or will the new assessments mark 
a big shift in your state/district? What coordination needs to occur between assessment and 
IT (information technology) departments to ensure that the technology in place meets assess- 
ment system demands and the needs of administrators and teachers? 

Data Managomont 

What systems do districts have that connect assessment results with professional development 
provided to administrators and teachers? What internal systems need to be developed to 
ensure that assessment data informs individualized professional development for teachers and 
individualized support for students? 

Alignment 

What are the best strategies for managing the transition to new standards and assessments while 
current NLCB-required assessments and accountability systems are still in place? How will 
states and districts align measures of teacher effectiveness to the new assessments? 

Enective collaboration 

What kind of coordination and communication currently exists between states and districts 
and districts and schools around issues of standards and assessments? Who is responsible 
for that coordination and communication? How will states, districts, and schools develop 
mechanisms for coordination and communication that facilitate flexible approaches to prob- 
lem-solving and addressing the inevitable challenges of implementing the new standards and 
assessments? Who needs to participate in planning for this work? 

Leaders who consider these issues now and forge new mechanisms for cross-system collaboration 
will be best-positioned to take advantage of the new CCSS and common assessments to improve 
student achievement. 
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for timeless values. 
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on the Wye River on Maryland’s Eastern Shore. Its international network includes partner Aspen 
Institutes in Berlin, Rome, Lyon, Tokyo, and New Delhi, and leadership programs in Africa and 
Central America. 



practitioners, researchers, and policy leaders to engage in focused dialogue regarding their efforts 
to improve student achievement, and to consider how public policy changes can affect progress. 
Through our meetings, analysis, commissioned work, and structured networks of policymakers 
and practitioners, the Program for nearly 30 years has developed intellectual frameworks on criti- 
cal education issues that assist federal, state, and local policymaking working to improve Ameri- 
can education. 
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